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ABSTRACT 

A study examined the effects of a college 
business/professional speaking course on the communication skills' 
development of students. The research examined three hypotheses: (1) 
that student ratings of their own communication skills would show a 
significant improvement after a 16”week course on business and 
professional speaking; (2) that males and females would not show a 
significant difference in their appraisal of their communication 
skills following the course; and (3) that undergraduate and graduate 
students would not show significant differences in their appraisal of 
their communication skills following the course. The study 
substantiated the first hypothesis but not the second or third. With 
regard to the first hypothesis , it was observed that when students 
viewed a videotape of their performance and then discussed it with a 
teacher their performance skills improved. With regard to the second 
hypothesis, male students did not report a significant improvement in 
their skills, while female students did. There are some indications 
that males are more developed in terms of their public speaking 
skills; if these indications are accurate, they could explain why 
males did not experience the degree of improvement that females did. 
With regard to the third hypothesis, graduate students did not report 
the degree of improvement that undergraduates did. These results 
could be explained by the fact that graduate students have more 
professional experience and educational background. (TB) 
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Abstract 



Only recently has the task at hand received tne attention it deserves. In 
the 1990s the issue has been raised on a regular basis by students, the faculty, 
school administrators, governing boards, legislators, and the public at large. 
This field study examined effects of a college business/professional speaking 
course on the communication skills development of students. The research focused 
on three hypotheses; HI; Student ratings of their communication skills 
development would show a significant improvement following a 16-week college 
course on business and professional spoaking, H2; No significant difference 
would exist between female and male student ratings of their communication 
skills development following participation in a 16-week college course on 
business and professional speaking, H3; No significant difference would exist 
between undergraduate and graduate student ratings of their communication skills 
development following participation in a 16-week college course on business and 
professional speaking. In relation to hypothesis 1 that student ratings of their 
communication skills development would show a significant improvement following 
a 16-week college course on business and professional speaking, the data showed 
at the .01 alpha level there was a statistically significant self-reported 
improvement. The second hypothesis that no significant difference would exist 
between female and male student ratings of their ccammunication skills 
development following participation in a 16-week college course on business and 
professional speaking was not upheld by the data. As it concerns hypothesis 3 
that no significant difference would exist between undergraduate and graduate 
student ratings of their communication skills development following 
participation in a 16-week college course on business and professional speaking, 
it was not upheld by the data. 
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ASSESSING STUDENT LEARNING OUTCOMES 
IN TEACHING BUSINESS AND 
PROFESSIONAL SPEAKING 

Introduction 

Today auninistrators and faculty are asked to provide fair, accurate, and 
consistent assessments of instruction. Initial considerations concerning 
evaluation of teaching effectiveness by faculty include; (a) Why do evaluations? 
(b) What are the purposes of evaluation? (c) Who should be evaluated? (d) What 
should be evaluated? and, (e) How should it be evaluated? 

Only recently has the task at hand received the attention it deserves. In 
the 1990s the issue has been raised on a regular basis by students, the faculty, 
school administrators, governing boards, legislators, and the public at large. 
Purpose Statement 

The purpose of this field research project was to assess the level of 
communication skills development reported by students enrolled in a college 
business and professional speaking course. 

Justification 

Since A Nation at Risk reached the bookshelves, there has been a national 
mandate for assessment in education in the United States. Angelo and Cross 
(1993) and others have focused on the question, how effectively are teachers 
teaching? In this paper we turn our focus to a specific dimension of teaching, 
that is the assessment of assessment of learning outcones in a business and 
professional speaking course. Appropriate assessment is important for two major 
reasons; (a) to insure student success in college and (b) to support faculty 
requests concerning promotion, tenure, and merit decisions. However, only 
recently has the task at hand received the attention it deserves. In the 1990s 
the issue has been raised on a regular basis by students, the faculty, school 
administrators, governing boards, legislators, and the public at large. 

Review of Literature 

In order to achieve the purpose of this field study, it was necessary to 
examine extant literature concerning (a) Importance of Business and Professional 
Speaking and (b) Assessment in Higher Education. 

Importance of Business and Professional Speaking 

Watt (1995) asked. Why is there so much attention given to business and 
professional speaking? Answer; Survival. As members of a democratic society, 
people must be able to effectively ccmmunicate. Sprague (cited in Carlson Se 
Smith-Howell, 1995 April) posited speech education should be a concern for all 
communication studies scholars and teachers. 

As suggested by VanDosting (1992), individuals cannot reach full potential 
unless they have reliable communication skills. He concluded the ability to 
effectively ccxtmunicate is the most important skills an individual possess. 
Assessment in Higher Education 

Identification of ways to effectively assess learning is critical in the 
field of communication studies (Aitken, 1994 November). Rubin, Welch, and 
Buerkel (1995, January) have noted that "in speech communication, teacher 
effectiveness has been measured typically by examining the teacher's classroom 
style and manner. . .observers or teachers report their own communication 
behaviors and researchers make judgments about qualities. ., .We suggest her that 
another method of assessing teacher impact is to determine the level of student 
development" (p, 30) . 
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Given the Importance to students and faculty, assessment of learning must 
meet a number of Important criteria in order to be effective. Experience has 
suggested the follov/ing eight principles apply when seeking to assess the 
effectiveness of learning in the classroom. 

Accuracy . When dealing with assessment, it is important to clearly and 
directly set forth the standards of measurement. Conformity to the established 
standards helps to assure an accurate assessment. This is possible because 
ambiguity has been reduced. 

Advocacy . It is important for the faculty to know the importance of 
teaching and its role in the process of educating students. Faculty's efforts 
need to be supported and recognized as the most important element of the 
profession of being a teacher. 

Clarity . Faculty need to "understand" v^at it means to "assess"; to be an 
"teach". Avoiding confusion and freeing them from ambiguity reduces tension on 
the part of the faculty member and allows greater performance. 

Consistency . Assessment ought to be consistent. Consistency allows for a 
feeling of stability. Varying expectations of the quality of teaching and the 
level of student learning among individuals should be avoided. It is imperative 
to set "expectations" and then to follow through as you assess the performance 
in the classroom. 

Delegation . Allow "sharing" in the decision-making process. Empower 
teachers to evaluate their work. Creating an environirent of "shared 
responsibility" can lead to a better quality of education. And, certainly 
faculty should be be encouraged to involve students in making decisions about 
course options and learning expectations. 

Fairness . Although it can be argued that "life isn't fair," that does not 
make it okay to give preferential treatment among those being evaluated in the 
classroom. Striving for objectivity during the assessment process is vital. 

Those assessing learning outcomes must be perceived as willing to protect the 
rights of both faculty and students, even if it means having to do so by 
overriding substantive organizational rules and procedures when it is in the 
best interest of the involved parties. 

Information. Information is crucial to effective teaching and learning in 
the classroom. Faculty and students should be "educated". Faculty must be given 
the information they need to teach effectively, and students ought to be able to 
expect that "appropriate" information will be given to them in a timely manner. 

Specificity. Teachers and students depend on the information they receive 
in order to make decisions, therefore it is necessary for both to be told 
exactly "how to" perform their tasks. 

Methodology 

This field study examined effects of a college business/professional 
speaking course on the communication skills development of students. The 
research focused on three hypotheses: 

HI: Student ratings of their communication skills development would show a 
significant improvement following a 16-week college course on business 
and professional speaking. 

H2: No significant difference would exist between female and male student 
ratings of their communication skills development following 
participation in a 16-week college course on business and professional 
speaking. 
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H3: No significant difference would exist between undergraduate and 
graduate student ratings of their communication skills development 
following participation in a 16-week college course on business and 
professional speaking. 

Participants 

The participants were undergraduate and graduate students in a fall 
semester course at a mid-sized, state university. They consisted of business 
communications, public relations, journalism, information networking and 
telecommunications, and general communication majors. There selected because 
they were enrolled in either CX)MM 514 or COMM 714 Business and Professional 
Speaking. 

Apparatus 

The Skills Assessment instrument was used as a pre- and post-test device 
for measuring the participants' level of development concerning 37 specific 
business/professional speaking skills. The instrument used was a 7-point Likert 
scale questionnaire wherein 1 designated "low development" through 7 for "high 
development" (see Appendix) . 

Procedure 

The following steps were used: 

Step 1. At the first class meeting of the fall semester the purpose of the 
field study was explained and the process was described to the participants 
(N=20 undergraduate and graduate students) enrolled in COMM 514 or COMM 714 
Business and Professional Speaking. Students were given an opportunity to 
transfer to another section of the same course. 

Step 2, Next students were asked to sign a "Agreement to Participate" form 
which included a protection of identity and confidentiality of information as 
well as a statement that only grouped data would be reported in the results. 
Participants were informed they could leave the study without penalty at anytime 
during the semester. 

Step 3. The pre-Skills Assessment instrument was distributed. Participants 
were asked to include their name, major, gender, and whether they were an 
undergraduate or graduate student. Students were given approximately 20 minutes 
to respond to the 37 items on the questionnaire. Once conpleted, the instrument 
was returned to the instructor of the course. 

Step 4. The 20 questionnaires were stored for safe-keeping in the 
department chair’s office in the communication department. 

Step 5, Over the next 15 weeks the participants received instruction and 
delivered 4 business/professional presentations. The required textbook was the 
courseworks edition of Theory and Applications for Effective Business and 
Professional Presentations (1995, revised edition) developed by Willis M. Watt 
and published by Brown and Benchmark Publishers. 

During the first 4 weeks students participated in lecture and discussion 
concerning several "traditional" topics dealing with business/professional 
speaking (e.g., researching, organizing, etc.) and conmunication concepts (e.g., 
communication apprehension, presentational delivery hints, etc.). This was 
followed by 4 laboratory experiences dealing with "instructional," "persuasive 
internal," "persuasive external," and "reportorial" presentations. Graduate 
students selected and completed a fifth presentation by choosing among the 
assignment options. 

Each "lab" experience was preceded by detailed explanation of the 
expectations of student outcomes and the evaluation. A sample video was 
presented for each type of presentation and using the course presentation 
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evaluation form, the students viewed and critiqued the sample. Each student's 
presentation v/as videotaped. After each student's presentation In the classroom, 
the student privately vlev«d and critiqued her or his own presentation while 
watching the videotape. N-V7t, the student and the Instructor met for a minimum 
of 30 minutes during which time they viewed the student's presentation and 
discussed Its strengths and weaknesses. Presentational development worksheets 
and outlines were also examined and reviewed dur.\ the teacher-student 
Interview period. 

Step 6 . During the class period In the final examination week, the 
participants (N=19 undergraduate and graduate students) completed the 
post-Skllls Assessment Instrument. The questionnaires were collected by the 
Instructor of the course. 

Step 7 . Data were then analyzed using descriptive statistics and the mean 
scores were analyzed using a two-sample, unmatched pairs t-test using Number 
Cruncher Statistical System (NCSS) by Hlntze (1992) . 



This field study examined effects of a college buslness/professlonal 
speaking course on the communication skills development of students. The 
hypotheses were: 

HI: Student ratings of their communication skills development would show a 
significant Improvonent following a 16-week college course on business 
and professional speaking. 

H2: No significant difference would exist between female and male student 
ratings of their communication skills development following 
participation In a 16-week college course on business and professional 
speaking. 

H3: No significant difference would exist between undergraduate and 
graduate student ratings of their ccanmunlcatlon skills development 
following participation In a 16-week college course on business and 
professional speaking. 

For the test the following data were collected: 

Table 1 

Gender and Level 



Results 



Males 



Females 



Undergraduate Graduate Undergraduate Graduate 



Pre-test (N=20) 



4 



3 



11 



2 



Post-test (N=19) 



3 



3 



11 



2 



One participant did not complete the post-Assessment Skills questionnaire. 
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Table 2 
Gender 





Males 


Females 




Mean Count 


Mean Count 


Pre-test (N=20) 


181 


162 


Post-test (N=19) 


223 


213 


T-values: Males -3.0807 


' Females -4.9093 




The data showed a 


statistically significant change at 


the .01 level for the 


female participants. A 
Table 3 

Education Level 


near significant value was reported for the males. 




Undergraduate 


Graduate 




Mean Count 


Mean Count 


Pre-test (N=20) 


161 


190 


Post-test (N=19) 


213 


222 



T-values; Undergraduate -5.6948 Graduate -1.9300 



The data showed a statistically significant change at the .01 level for the 
undergraduate participants. A non-significant value was reported for the 
graduates. 
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Table 4 

Overall Skills Envelopment 



Mean 

Pre-test (N=20) 

Post-test (N=19) 

T-value: -5.6349 

The data showed a statistically significant change at the .01 level for the 
participants. 

Figure 1 

Line Plot - Femal e 

Score 93 

Pre-test 1.. 

Post-test ... 

R=93-237 
Figure 2 

Line Plot - Male 

Score 141 239 

Pre-test 1 1..11 1 1 1 

Post-test 1 1...1 3... 



237 



. 11 . 



. 1 .. . 1 , 1 . 1.2 2 ... 1..1 

1 . 1 . . 1 . 11 . 1 . 1 . 



,2211 



169 

216 



R=141-239 
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Figure 3 



Line Plot ■ 


- Undergraduate 


Score 


93 235 


Pre-test 


1... 11 1...1..21111»...11..2 


Post- test 


1..1..111..1.1 223 



R=93-235 



Figure 4 




Line Plot 


- Graduate 


Score 


141 240 


Pre-test 




Post-test 


1 1..1 1.1 



R=141-240 



Figure 5 




Line Plot 


Overall 


Score 


93 237 


Pre-test 


1 11.. 1.1.. .1.2.212 2.. 11.. 1.1 1 


Post-test 


..1.1. .1111.1.1. .1.1. .2331. 


R=9 3-237 





Discussion 



Having implemented the methodology and analyzed the results, it was 
necessary to examine a number of conclusions concerning the hypotheses and 
assessment, explore the limitations of this field research, and offer some 
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suggestions for further research. 

Conclusions 

In relation to hypothesis 1 that student ratings of their communication 
skills development would show a significant improvement following a 16-week 
college course on business and professional speaking, the data showed at the .01 
alpha level there was a statistically significant self-reported improvement. 

This improvement is likely to have occurred as a result of the laboratory 
experiences (4 for undergraduate; 5 for graduate). Over the years an intuitive 
observation has been when students rate their own performance by watching a 
videotape followed by an interview with the teacher to observe and discuss their 
performance, the students have experienced improved presentational skills. The 
results of this assessment uphold such a conclusion. Also, it has been felt that 
by observing and rating a sample presentations using the course evaluation form 
the students presentational development is enhanced. Once again, this conclusion 
appears to be supported by the assessnent data gathered in this study. 

The second hypothesis that no significant difference would exist between 
female and male student ratings of their communication skills development 
following participation in a 16-week college course on business and professional 
speaking was not upheld by the data. The results reflected no significant gain 
reported by the male students (although it was near significance). On the other 
hand, the female subjects reported a statistically significant improvement in 
their presentational skills. Sane indications have suggested that males are 
generally more developed in terms of public address skills. If this is accurate, 
it would seem reasonable for the females, who traditionally have been given less 
attention and fewer opportunities to develop presentational skills, would report 
a greater degree of skills development than their male counterparts. 

As it concerns hypothesis 3 that no significant difference would exist 
between undergraduate and graduate student ratings of their communication skills 
development following participation in a 16-week college course on business and 
professional speaking, it was not upheld by the data. As reported in Table 3 and 
shown in Figures 4 and 5, the undergraduate participants reported statistically 
significant improvement in their conmunication skills development, however, no 
such repxjrt was made by the graduate students. Two factors may have influenced 
these results, in that: (a) the graduate students have more professional 
experience and (b) they have more educational experience. 

Finally, concerning the broader issue of assessment as discussed above, 
assessment is important for two major reasons: (a) to insure student success in 
college and (b) to support faculty requests concerning promotion, tenure, and 
merit decisions. Assessment has been receiving a lot of attention in the 1990s. 
The issue has been raised on a regular basis by students, the faculty, school 
administrators, governing boards, legislators, and the public at large. This 
field research provides factual data these parties can use in assessing teaching 
effectiveness and learning outcomes. In fact, many, if not all, of the criteria 
of effective assessment previously discussed can be met when such an assessment 
technique is employed in the classroom. Concerned parties should be encouraged 
to employ this or similar strategies. 

Limitations and Further Research 

Certainly the data is useful, however, the sample was limited in size 
(pre-test N=20; post-test N=19) . This resulted in small cells for conparisons. 
The study should be replicated several times in order to enhance the quality of 
the data. 

Also, the assessment tool involved a "self-repxort" . Such data is less 




11 



Page 9, BPS Assessment 

reliable than observational and experinental data. It would be beneficial to 
conduct the study in a more controlled environment in order to show more 
conclusively the effects of the course on communication skills development for 
business and professional speaking. 
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APPENDIX A 



"Skills Assessment Form" 



I 

tne nu.T.i 



.'12: For each of the skills identified in t.he list be^cw, circle 

a^- which best describes ycur current level of skill development. 



LOW 3E7EL2FME.iT 



HIGH 0E7EL0PME.NT 



Ur.darstandi ng mean'ngs 
Analyze while listening 
Give feectack 
Ask questions 

C'.iocse a specific speech topic 
Choose a specific speech purpose 
Develop main points 
Organize the main points 
Analyze the audience 
Research a speech topic 
Support main points 
Develop logical arguments 
Develop emotional appeals 
Develop ycur credibility 
Develop an introduction 
Develop a conclusion 
Develop an outline 
Use visual aids 
C, noose words 

Handle communication apprehension 
Be assertive 

Articulate and pronounce words correctly 

Practice a speech 

Remember a speech 

Deliver a speech 

Think on your feet 

Plan for audience response 

Adapt to feedback from listeners 

Adapt to the speaking situation 

Handle questions from the audience 

Handle criticism 

Determine if you accomplished your goal 
Identify areas for further development 
Change audience's attitudes 
Change audience's actions 
talk on short notice 
Handle special occasion speaking 
si tuations 



1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

]_ 

I 

1 

1 

1 

i 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 



2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

c 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 

2 



3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 

3 



4 5 
4 5 
4 . 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 

4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 5 
4 'S 
4 5 
4 5 
4 5 
4 5 

4 5 



6 

5 

6 
6 
6 
6 
6 
6 
6 

5 

6 
6 
6 
6 

6 

6 

6 

6 

6 

6 

5 



6 

6 

6 

6 

6 

6 

6 

6 

6 

•6 

6 

6 

6 

6 



6 



7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 

7 



NOTE- This form will be kept on file until the end of the semester at which 
time you will be asked to reassess your public speaking s<ills. 
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